Skip to content

fix(uffd): dedupe deferred page faults#2864

Open
ValentaTomas wants to merge 11 commits into
mainfrom
valenta/uffd-dedupe-deferred-eagain
Open

fix(uffd): dedupe deferred page faults#2864
ValentaTomas wants to merge 11 commits into
mainfrom
valenta/uffd-dedupe-deferred-eagain

Conversation

@ValentaTomas
Copy link
Copy Markdown
Member

Summary

Deduplicate deferred UFFD faults by address so repeated EAGAINs for the same page do not flood the retry queue or wake loop.

Test plan

  • Not run; Linux UFFD package depends on Linux cgo types and cannot execute on this macOS host.

@cla-bot cla-bot Bot added the cla-signed label May 29, 2026
@cursor
Copy link
Copy Markdown

cursor Bot commented May 29, 2026

PR Summary

Low Risk
Small, localized change to deferred fault retry bookkeeping in the Linux sandbox UFFD path, with unit tests and no auth or data-path impact.

Overview
Deferred userfaultfd retries now keep at most one queued fault per page address, so many threads faulting the same page after EAGAIN no longer multiply entries in the retry queue and poll wake loop. When a later fault for that page is a write, the kept entry is upgraded to a write fault so the retry resolves the page dirty instead of relying on a follow-up WP fault.

Reviewed by Cursor Bugbot for commit 2b7e091. Bugbot is set up for automated code reviews on this repo. Configure here.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
2707 2 2705 5
View the full list of 2 ❄️ flaky test(s)
github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity

Flake rate in main: 57.71% (Passed 740 times, Failed 1010 times)

Stack Traces | 69s run time
=== RUN   TestSandboxMemoryIntegrity
=== PAUSE TestSandboxMemoryIntegrity
=== CONT  TestSandboxMemoryIntegrity
    sandbox_memory_integrity_test.go:27: Build completed successfully
--- FAIL: TestSandboxMemoryIntegrity (68.98s)
github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity/tmpfs_hash

Flake rate in main: 57.83% (Passed 730 times, Failed 1001 times)

Stack Traces | 199s run time
=== RUN   TestSandboxMemoryIntegrity/tmpfs_hash
=== PAUSE TestSandboxMemoryIntegrity/tmpfs_hash
=== CONT  TestSandboxMemoryIntegrity/tmpfs_hash
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{start:{pid:1259}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Total memory: 985 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Used memory before tmpfs mount: 193 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Free memory before tmpfs mount: 791 MB\nMemory to use in integrity test (60% of free, min 64MB): 474 MB\n"}}
Executing command bash in sandbox iv6onr1rvybod2v97dlph (user: root)
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"474+0 records in\n474+0 records out\n497025024 bytes (497 MB, 474 MiB) copied, 2.16223 s, 230 MB/s\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Used memory after tmpfs mount and file fill: 669 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"\tCommand being timed: \"dd if=/dev/urandom of=/mnt/testfile bs=1M count=474\"\n\tUser time (seconds): 0.00\n\tSystem time (seconds): 1.99\n\tPercent of CPU this job got: 91%\n\tElapsed (wall clock) time (h:mm:ss or m:ss): 0:02.16\n\tAverage shared text size (kbytes): 0\n\tAverage unshared data size (kbytes): 0\n\tAverage stack size (kbytes): 0\n\tAverage total size (kbytes): 0\n\tMaximum resident set size (kbytes): 2636\n\tAverage resident set size (kbytes): 0\n\tMajor (requiring I/O) page faults: 3\n\tMinor (reclaiming a frame) page faults: 344\n\tVoluntary context switches: 4\n\tInvoluntary context switches: 45\n\tSwaps: 0\n\tFile system inputs: 176\n\tFile system outputs: 0\n\tSocket messages sent: 0\n\tSocket messages received: 0\n\tSignals delivered: 0\n\tPage size (bytes): 4096\n\tExit status: 0\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_memory_integrity_test.go:70: Command [bash] completed successfully in sandbox ihllff4vv3buv3t5d7ky1
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{start:{pid:1275}}
Executing command bash in sandbox i3j7ic510elbcxuk4gsup (user: root)
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{data:{stdout:"76a021f37a9659849c0e0c3697177da8237a2e38edb2d04d66382f4b625c6d16\n"}}
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_memory_integrity_test.go:80: Command [bash] completed successfully in sandbox ihllff4vv3buv3t5d7ky1
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{start:{pid:1278}}
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
Executing command bash in sandbox ihllff4vv3buv3t5d7ky1 (user: root)
    sandbox_memory_integrity_test.go:110: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:81
        	            				.../hostedtoolcache/go/1.26.3.../src/runtime/asm_amd64.s:1771
        	Error:      	Received unexpected error:
        	            	failed to execute command bash in sandbox ihllff4vv3buv3t5d7ky1: unavailable: HTTP status 502 Bad Gateway
    sandbox_memory_integrity_test.go:110: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:78
        	            				.../tests/orchestrator/sandbox_memory_integrity_test.go:110
        	Error:      	Condition never satisfied
        	Test:       	TestSandboxMemoryIntegrity/tmpfs_hash
--- FAIL: TestSandboxMemoryIntegrity/tmpfs_hash (198.86s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces deduplication of deferred pagefaults by address in the deferredFaults struct to prevent redundant wakeups. It adds a byAddr map to track and deduplicate addresses, updates the Serve loop to only signal wakeups when a new pagefault is successfully deferred, and includes unit tests to verify this behavior. There are no review comments, and no issues were identified in the changes.

@ValentaTomas ValentaTomas marked this pull request as ready for review May 29, 2026 22:04
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1824ece8d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/orchestrator/pkg/sandbox/uffd/userfaultfd/deferred.go Outdated
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Comment thread packages/orchestrator/pkg/sandbox/uffd/userfaultfd/deferred.go Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 306359e1ab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/orchestrator/pkg/sandbox/uffd/userfaultfd/deferred.go
Drop the dedupe map after draining so one burst of unique addresses does not retain map capacity for the sandbox lifetime.
Key deferred page faults by page boundary so same-page offsets coalesce before retry.
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dd691d5002

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/orchestrator/pkg/sandbox/uffd/userfaultfd/deferred.go Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants